This invention relates to clock systems using crystal oscillators, and in particular to such systems employing numerous crystal oscillators in a synchronized clock system.
Clock signals are the most important control signals in digital systems. Regardless of the modulation scheme, the timing of logic transitions is dictated by the system's clock. The performance of a system at any level—within a core or chip, across cores on an integrated chip, across chips on a board, or across boards—is predicated by the coordination of clock signals among the components. Example known applications can be described as follows.
Synchronous Systems. Synchronous systems provide clock signals that are frequency locked at every transmission/reception exchange and require a design with zero skew to set the phase relationship between signal and clock. Difficulties in coordinating clock signals in synchronous systems differ at the chip, circuit, and system levels. On a chip, a single clock is easily distributed to drive every element and data-clock skew is easily controlled. The situation is more complicated at the circuit (multi-chip) and system (multi-board) levels. Synchronization requires that either (1) a central clock be distributed across the circuit, (2) independent clocks local to circuit components be frequency locked, or (3) a low frequency reference clock be distributed across the circuit and multiplied up to the data rate at each component. Each of these solutions used in current systems introduces another layer of problems in component cost, design complexity, increased jitter and noise, and reduced reliability. Additionally, the difficulties increase further with component count and separation distance.
In an ideal synchronous circuit, every change in the logic levels of every component is defined by the level change of a common clock signal simultaneously; the timing of all events can be safely assumed, and there is no need for active components to monitor and coordinate the timing of different events. The real world, however, is nothing like the ideal. In real circuits, logic transitions have finite rise/fall times, signals take time to propagate, and registers have nonzero latch times. All of these factors and more combine to dictate the maximum possible system speed. At the chip level, the combination of the clock quality and the delay of each component on a chip set a limit on the maximum clock speed. At the circuit and system levels, things are different, all events may not be simultaneous, but conventionally the timing of every event needs to be in some way coordinated at the system level nonetheless. In inter-board systems (e.g. server blade applications), a module operating in one clock domain inevitably needs to send data to another module operating in a second clock domain
Currently the most common synchronous systems distribute a common clock signal by fanning out a master clock to each component of the system. A single input clock signal is re-driven by several output buffers. The buffers have propagation delay, though fan-outs are available that incorporate phase-locked loops (PLLs) to eliminate skews between the outputs. However, PLLs introduce jitter. When more than one fan-out part is required, it is important to include adjustable delay in the circuit to eliminate skew between fan-out models. In many current applications, a low frequency clock is fanned out across a system and the clock is multiplied to the data rate at each component. The jitter of the PLL multiplier's Voltage Controlled Oscillator (VCO) is added to the clock signal and, as a result of multiplication, the jitter of the clock itself increases as the square of the multiplication factor.
Another current technique for clock distribution is to simply daisy chain a single clock signal across the system. At each component, a well-tuned delay must be provided to synchronize the system. In practice, it is difficult to match impedances so perfectly that the clock signal isn't reflected at each tap. Multiple reflections interfere with the signal and introduce noise and jitter.
Skew is the fixed timing between two signals. The primary cause of skew is the difference in trace length, but anything that affects signal propagation can contribute: trace width and impedance, variations in dielectric constants, and temperature. If the receiver samples the data on the rising edge of the clock signal, then as long as the clock provides the receiver a rising edge at the right time, there is no relevant skew. However, taking jitter into account, assuring that the same clock edge that was used to generate a data transition is also used to strobe that transition at the receiver can dramatically decrease the effective jitter of the system. If the data system and the clock signal both have the same jitter, they can trace each other. Ensuring that the clock used in a receiver has the same jitter as the data is one of the driving motivations for adopting asynchronous architectures.
Asynchronous Systems. Asynchronous systems have more autonomous components than synchronous systems; they are not frequency-locked or phase-locked and, between components, delay and skew are not an issue. At the transmitter, the clock signal determines logic transitions and, at the receiver, rather than simple incoming data with the trivial assumption of synchronous timing, a separate clock must be at least temporarily phase-locked and frequency-locked so that bits can be sampled at their centers.
Current asynchronous architectures have several advantages over synchronous designs at the inter-board level, few advantages at the circuit level, and, except in the most rare cases, no advantages at the chip level. Asynchronous systems solve several of the problems presented by common synchronous systems: fan-out and the associated increased jitter does not present a problem, skew is not a problem, and having multiple clocks reduces the possibility of catastrophic central clock failure. The autonomous nature of asynchronous architecture provides scalability and redundancy. The reduced coordination between boards provides for easier addition and subtraction as needed.
However, different components must still communicate in the systems, and to do so, an element of synchronization is required. The first sacrifice made in moving from a synchronous to an asynchronous architecture is the seamless transparent timing of each event in the system. This amounts to surrendering the ultra-high performance that can only be attained in a system where every event occurs in harmony. One way to achieve the level of synchronization necessary for communication in an asynchronous system is to have the transmission of a data signal controlled by one clock, and its reception controlled by another. Another way is to use a clock recovery system. Here, the VCO of a PLL is locked to the transitions of the incoming data and is used to strobe the receiver; the clock used to reconstruct incoming data is embedded in the data itself. Other than within the clock recovery circuit where the positioning of the strobe must be well centered in the setup and hold comfort zone of the receiver, problems with skew are eliminated. The wider the bandwidth of the clock recovery circuit, the more that jitter on the clock tracks jitter on the data. In some designs, a low frequency clock signal is distributed to the receiver to aid the clock recovery circuit. PLL-based clock recovery circuits are expensive components, and the digital alternative, a Phase Interpolator (PI), is less expensive but more difficult to characterize. PIs are also more likely to suffer nonlinear effects and usually require a distributed clock.
The general concept of having two synchronization busses and connecting clock modules to them alternatingly was described in The Future of Multi-Clock Systems, by Ransom Stephens, Roman Boroditsky and Jorge Gomez, DesignCon 2008. That paper described a synchronous clock circuit with the Sync In of a first SXO module being connected to the Sync A bus, the Sync Out of the first SXO module being connected to the Sync B bus, the Sync In of a second SXO module being connected to the Sync B bus, the Sync Out of the SXO second module being connected to the Sync A bus.
This invention relates to improvements to the systems described above, and to solutions to some of the problems raised or not solved thereby.